iReasearch学术科研网

摘要：In a recent state-of-the-art (SoA) article (Stoynoff 2009), I reviewed some of the trends in language assessment research and considered them in light of validation activities associated with four widely used international measures of L2 English ability. This Thinking Allowed article presents an opportunity to revisit the four broad areas of L2 assessment research (conceptualizations of the L2 construct, validation theory and practice, the application of technology to language assessment, and the consequences of assessment) discussed in the previous SoA and to propose tasks I believe will promote further advances in L2 assessment. Of course, the research tasks I suggest represent a personal stance and readers are encouraged to consider additional perspectives, including those expressed by Bachman (2000), Chalhoub-Deville & Deville (2005), McNamara & Roever (2006), Shaw & Weir (2007), and Stansfield (2008). Moreover, readers will find useful descriptions of current research approaches to investigating L2 assessments in Lumley & Brown (2005), Weir (2005a), Chapelle, Enright & Jamieson (2008), Lazaraton (2008), and Xi (2008). Adapted from the source document

摘要：Few prior studies have examined degree of fidelity between raters' assessments of oral performances and objectively observable prosodic indices of accentedness. Prosodic indices of accentedness quantify trait-relevant variance, whereas rater background variables represent trait-irrelevant variance. The present study, therefore, investigated the extent to which raters' background characteristics and instrumentally measured prosodic indices of speakers' accentedness jointly influenced the rating of oral performances. Seventy U.S. undergraduate students rated the speaking and teaching proficiency of 11 international teaching assistants (ITAs). Using the PRAAT computer program, 5 min of continuous speech from each of the ITAs were instrumentally analyzed for a number of indices of speech rate, pausing, stress, and intonation. Dependent variables were undergraduates' ratings of ITA oral proficiency and instructional competence. Rater background variables such as the listener's native speaker status and experience as a language tutor explained 7-9% of the variance in oral performance ratings, whereas 18-19% was attributable to the prosody variables. These findings suggest that U.S. undergraduates are sensitive to trait-relevant indicators of ITA oral proficiency. At the same time, their speech evaluations are subject to substantial bias based on their own backgrounds. Adapted from the source document

摘要：This study investigates aspects of validity of an alternative measure of productive vocabulary. Lex30, developed by Meara and Fitzpatrick, is a word association task that claims to give an indication of productive vocabulary knowledge. Previous studies of Lex30 have assessed test-retest reliability, performance against native speaker norms, concurrent validity, reliability of parallel forms, and ability to reflect improvements in vocabulary development. In addition, the issue of construct validity has been explored. The study described here replicates some of these investigations with a different population and extends the investigation of construct validity. By comparing the performance of second language (L2) learners at different proficiency levels, the ability of the test to distinguish between levels of proficiency is explored. Concurrent validity is explored by comparing L2 learners' performance on Lex30 with that of two other productive vocabulary tests. Finally, one aspect of construct validity is explored by assessing whether Lex30 measures productive vocabulary use or simply recall. The findings indicate that Lex30 is a reliable and valid measure of productive vocabulary knowledge, but whether it measures only recall, or whether it measures actual ability to use vocabulary meaningfully and appropriately, appears to depend on the proficiency level of the test taker. Adapted from the source document

摘要：The Assessment of Language Competence (ALC) certificates is an annual, international testing program developed by the Australian Council for Educational Research to test the listening and reading comprehension skills of lower to middle year levels of secondary school. The tests are developed for three levels in French, German, Italian and Japanese, and at two levels in Chinese and Indonesian. There is a mixture of target language and English questions in the Level 2 and 3 tests. Some teachers have raised this as a concern in the belief that all questions should only be offered in English for the sake of fairness. Their view is that the tests are unduly difficult when they are designed with questions in the language. Arising from this concern, the aim of the research to be reported in this paper was to investigate the effects of the language of the question on student performance. We drew on data from a trial test and a final listening test, a questionnaire administered to students to gauge their perceptions of the tests and an examination of public documentation about the ALC. For the statistical analysis, we used Item Response Theory for calibrating items and for comparing item difficulty estimates, and fit statistics to verify how well items with English and target language question formats worked together. We found that where the questions involved listening for simple, explicitly stated information, students found the items in the target language relatively easier. In questions that required students to listen for global meaning, language choice either did not matter or tended to favour items in English rather than the target language. Furthermore, each of the six tests with some items in English and others in the target language showed a high level of reliability and fit to the single latent scale, indicating that items were functioning consistently regardless of the language of the test question. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：This paper presents a comprehensive framework for researching classroom-based assessment (CBA) processes, and is based on a detailed empirical study of two Australian school classrooms where students aged 11 to 13 were studying Indonesian as a foreign language. The framework can be considered innovative in several respects. It goes beyond the scope of earlier models in addressing a number of gaps in previous research, including consideration of the epistemological bases for observed assessment practices and a specific learner and learning focus. Moreover, by adopting the broadest possible definition of CBA, the framework allows for the inclusion of a diverse range of data, including the more intuitive forms of teacher decision-making found in CBA (Torrance & Pryor, 1998). Finally, in contrast to previous studies the research motivating the development of the framework took place in a school-based foreign language setting. We anticipate that the framework will be of interest to both researchers and classroom practitioners. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：This paper presents a framework for incorporating the assessment of psycholinguistic constructs into spoken language proficiency testing. Although response time and automaticity of language processing are vital factors in language acquisition, as constructs they are rarely explicitly assessed in language tests. It is proposed that efficiency of processing, although it does not appear in most models of communicative language competence, might be viewed as a language competence in its own right. In contrast to communicative approaches to testing which evaluate mastery of complex language structures or domain-specific vocabulary, an approach which emphasizes efficiency of processing evaluates the speed and accuracy with which a learner processes familiar language, as this provides equally useful information about their L2 proficiency. In a study to demonstrate this, the speech of 147 test takers was evaluated as they performed on conceptually simple and straightforward performance tasks such as elicited imitation. It was found that sentence repetition and sentence building tasks reliably separated test takers according to proficiency in a way that accounts for the trade-off between fluency, accuracy and complexity. While these tasks would not be considered communicative, they are psycholinguistically valid and also link to good practice in classroom language learning activities. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：This study explores how second language (L2) texts written by learners at various proficiency levels can be classified using computational indices that characterize lexical competence. For this study, 100 writing samples taken from 100 L2 learners were analyzed using lexical indices reported by the computational tool Coh-Metrix. The L2 writing samples were categorized into beginning, intermediate, and advanced groupings based on the TOEFL and ACT ESL Compass scores of the writer. A discriminant function analysis was used to predict the level categorization of the texts using lexical indices related to breadth of lexical knowledge (word frequency, lexical diversity), depth of lexical knowledge (hypernymy, polysemy, semantic co-referentiality, and word meaningfulness), and access to core lexical items (word concreteness, familiarity, and imagability). The strongest predictors of an individual's proficiency level were word imagability, word frequency, lexical diversity, and word familiarity. In total, the indices correctly classified 70% of the texts based on proficiency level in both a training and a test set. The authors argue for the applicability of a statistical model as a method to investigate lexical competence across language levels, as a method to assess L2 lexical development, and as a method to classify L2 proficiency. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：This study examines the associations between the speaking proficiency of 181 adult learners of Dutch as a second language and their linguistic competences. Performance in eight speaking tasks was rated on a scale of communicative adequacy. After extrapolation of these ratings to the Overall Oral Production scale of the Common European Framework of Reference for Languages (CEFR) (Council of Europe, 2001), 80 and 30 participants (on average per speaking task) were found to be, respectively, at the B1 and B2 levels of this scale. The following linguistic competences were tapped with non-communicative tasks: productive vocabulary knowledge, productive knowledge of grammar, speed of lexical retrieval, speed of articulation, speed of sentence building, and pronunciation skills. Discriminant analyses showed that all linguistic competences, except speed of articulation, discriminated participants at the two levels of oral production. Subsequent comparisons showed that the distance between B1ers and B2ers was smaller in knowledge of high-frequency words than in knowledge of medium- and low-frequency words. Extrapolation from scores on the vocabulary test yielded estimations of productive vocabularies of, on average, 4000 and 7000 words for B1ers and B2ers, respectively. The grammar test assessed grammatical knowledge in 10 domains. B2ers were found to outperform B1ers on all parts of the test. Thus, the differences in lexical and grammatical knowledge of B1ers and B2ers appear to be a matter of degree, rather than a matter of category or domain. The paper ends with a research agenda for a linguistic underpinning of the CEFR. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：The Modern Language Aptitude test, which measures the test-taker's ability to learn a foreign language, remains widely used for practical and research purposes despite the fact that its single original form has remained unchanged since its first release in 1959. This article examines the test in historical perspective, as Sasaki comments on its past, present, and future. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：The fuzziness of assessing second language speaking performance raises two difficulties in scoring speaking performance: indistinction between adjacent levels and overlap between scales. To address these two problems, this article proposes a new approach, confidence scoring, to deal with such fuzziness, leading to confidence scores between two adjacent levels applied to three scales. Since confidence scores have to be transformed to an exact score for test interpretation and use, membership functions and rule bases are applied and a confidence scoring algorithm is developed. Confidence scoring is demonstrated in the paper by an example to facilitate easy understanding. The paper then describes a pilot study that was conducted to try out the confidence scoring design. Initial results reveal that: first, confidence scoring is as feasible as traditional scoring; second, confidence scoring performs better in scoring dependability and in correlations with established benchmarks. At the end of the article, further studies are called for in order to build a validity argument and make further revisions to the confidence scoring method described here. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：This study investigated the relationships among the quality of L2 test takers' notes evaluated in terms of different levels of information and test takers' performance on open-ended listening tasks tapping into different comprehension subskills. In addition, this study examined the invariance of the structural relationships among the variables across two different note-taking formats, that is, a blank format and an outline format, by employing a multi-group structural equation modeling (SEM) approach. The results indicated that note quality measures, in particular the number of topical ideas found in the notes and the organization of these notes, may be good indicators of test takers' second language academic listening proficiency. It was also found that despite the invariance of structural relationships among variables across the two note-taking formats, the associations between the open-ended listening measures and note quality measures were slightly stronger in the outline format than in the blank format. The implications of these results for L2 academic listening assessment are considered. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：Testing of second language pragmatic competence is an underexplored but growing area of second language assessment. Tests have focused on assessing learners' sociopragmatic and pragmalinguistic abilities but the speech act framework informing most current productive testing instruments in interlanguage pragmatics has been criticized for under-representing the construct. In particular, the assessment of learners' ability to produce extended monologic and dialogic discourse is a missing component in existing assessments. This paper reviews existing tests and argues for a discursive re-orientation of pragmatics tests. Suggestions for tasks and scoring approaches to assess discursive abilities while maintaining practicality are provided, and the problematicity of native speaker benchmarking is discussed. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：With the increasing use of integrated tasks in assessing writing, more and more research studies have been conducted to examine the construct validity of such tasks. Previous studies have largely focused on reading-writing tasks, while relatively little is known about graph-writing tasks. This study examines second language (L2) writers' test-taking strategies in relation to their performance on a graph-writing test administered to English learners in health science and medical majors. Data from a strategy inventory and open-ended questions were collected and analyzed to determine if the task elicited academic writing strategies and to identify construct-related issues. The results from structural equation modeling analyses indicated that writers were engaged in graph comprehension, graph interpretation, and graph translation strategies during the task, and each type of strategy use generally had a positive impact on their test performance. The qualitative results pointed to L2 writers' difficulties in lexical knowledge and use, as well as possible sources of construct-irrelevant variance including graph familiarity, topical knowledge, and test-wiseness strategy use. The findings have theoretical and practical implications for the development and use of graph-writing tasks. [Copyright The American University; published by Elsevier Ltd.]

摘要：This article reports on a replication study of a substantial piece of research published in The Modern Language Journal, carried out by Elaine Matties Phillips in 1992, and entitled "The Effects of Language Anxiety on Students' Oral Test Performance and Attitudes." The aim of both studies was to assess the influence of language anxiety experienced by university foreign language students on their performance in an oral examination. In both investigations, Pearson correlations and partial correlations were conducted between oral exam grades and the Foreign Language Classroom Anxiety Scale (FLCAS; Horwitz, Horwitz, & Cope, 1986) and between 8 oral performance criteria and the FLCAS. Analyses of variance were used to evaluate relationships between speaking performance and language anxiety at 3 levels (low, moderate, and high). Individual interviews were held with selected highly anxious students immediately after the oral exam. By replicating Phillips's study, the present researchers confirmed certain aspects of her outcomes and improved on it in various ways. Additionally, some differing results came to light. Adapted from the source document

摘要：Abstract not available.

摘要：In an investigation with advanced language learners, Brantmeier [Brantmeier, C., 2006. Advanced L2 learners and reading placement: self-assessment, computer based testing, and subsequent performance. System 34 (1), 15-35.] reports that self-assessment (SA) of second language (L2) reading ability, when measured with self-rated scales, is not an accurate predictor of subsequent reading performance as measured via multiple choice items. In another experiment with advanced learners that utilizes criterion-referenced SA items, Brantmeier and Vanderplank [Brantmeier, C., Vanderplank, R., 2008. Descriptive and criterion-referenced self-assessment with L2 readers. System 36 (3), 456-477] reveal that learners accurately estimate their reading comprehension when it is measured via multiple choice items. For the present study, an SA instrument of language learning achievement was designed according to specific course content to take into consideration the direct experience learners have had in practicing reading, listening, speaking, and writing [Ross, S., 1998. Self-assessment in second language testing: a meta-analysis of experimental factors. Language Testing 15, 1-20.]. With 276 participants, the study examines skill-based SA across beginning, intermediate and advanced levels of language instruction, and it offers evidence to validate the relationship between the SA instrument and achievement on an online abilities test with advanced learners. Findings hold important implications for language learner assessment, especially in terms of advanced students' ability to rate themselves when given specific criteria. A discussion about the value of SA as a complement to other traditional approaches for language program assessment is offered. [Copyright Elsevier Ltd.]